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Abstract. A description is given of the image of the Weil rep- 
resentation of the symplectic group in the Schwartz space and in 
the space of tempered distributions under the Gaussian integral 
transform. We also discuss the problem of infinite dimensional 
generalization of the Weil representation in the Schwartz space, in 
order to construct appropriate quantization of free scalar field. 



1. Introduction 

This work arose during the study of the problem of quantization of 
fields, i. e., mathematically and logically self-consistent construction 
of quantum field theory. One of formulations of this problem is to 
give a mathematical sense to the quantum field theory Schrodinger 
variational differential equation [3,8] and its relativistically invariant 
generalization [13-15]. 

The Schrodinger equation for free scalar field reads 

.,9* f ( h 2 5 2 I. , . , N2 m 2 . , 2 \ T , 

(i) *_=y (- T ^ + 2fe-d«(x)) 2 + -« ( x)^^ 

Here \I/ is an unknown functional depending on a number t and on a 
real function u(x), x = (x\, . . . , Xk); is the variational derivative. 

Traditionally one solves equation (1) in the Fock Hilbert space con- 
taining functionals of the form 

= * («) exp (—^ J u(p)u(-p)ujpdp 

where \l/o is a polynomial functional of u; p = (pi, . . . ,pk), u{p) = 
( 27r )fc/2 / e ipx u(x)dx, uj p = \J p 2 + m 2 . It is easy to see that on these 
functionals the right hand side of equation (1) equals infinity. To over- 
come this, one subtracts an "infinite constant" from the Hamiltonian 
in the right hand side of (1), and reduces this Hamiltonian to a normally 

ordered expression of creation operators -k — (j~h s ^_ p ^ + uj p u(p)j and 
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annihilation operators — ^= — ^/i gfl(_ p ) + ^^(p)) • This approach meets 

big problems [8]. One of them is that in the Fock space for k > 1 
one cannot give a mathematical sense to relativistically invariant gen- 
eralization of equation (1), i. e., one cannot perform quantization on 
spacelike surfaces [16]. 

The idea of our approach to equation (1) was to try to define an 
analog of the space of main functions and distributions on the infi- 
nite dimensional space of functions u(x) (let us call them main and 
distribution functionals), and to solve equation (1) in these distribu- 
tion functionals. This approach is also valid from the physical point of 
view (unlike the approach of considering dynamics in Hilbert space), 
since quantum mechanical quantities like energy and momentum are 
non-measurable in relativistic quantum dynamics, and the only mea- 
surable quantities are the scattering sections. A negative result of the 
present paper is that this approach turned out to be not fruitful for the 
problem of giving mathematical sense to equation (1) and quantization 
on spacelike surfaces (see Remark 7 at §6), but it led to some math- 
ematical results which we also present in this paper. The problem of 
quantization of a free field on spacelike surfaces is solved in the paper 
[18]. 

The space of main functionals that we look for could satisfy the 
following requirements. 

1) It is a locally convex topological vector space with the action 
of differential operators with polynomial coefficients, or rather with 
the action of infinite dimensional analog of the Weyl algebra (cf., for 
example, [17], §18.5). 

2) It has an action of infinite dimensional symplectic group. (This 
is needed, for example, for quantization on spacelike surfaces. Indeed, 
in classical field theory the evolution operator of the Klein-Gordon 
equation from one spacelike surface to another one is a linear symplectic 
transformation of the phase space of a free field.) 

3) It has the distribution functional 1. 

4) Its finite dimensional analog is the Weil representation of the 
symplectic group Sp(2n, R) in the Schwartz space S(M. n ) of rapidly 
decreasing smooth functions ([5,10]; [17], §18.5; see §2 below). 

5) It contains Gaussian functionals, i. e., exponents of a quadratic 
form. 

Let us comment on the last requirement. In the finite dimensional 
case Gaussian functions are transformed under the action of the sym- 
plectic group in the most simple way. Hence one can expect that in 
the infinite dimensional case main functionals will also contain analogs 
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of Gaussian functions. Besides that, the space of asymptotic states 
as t — > ±00 should be identified with the Fock space, to make the 
S'-matrix a unitary operator in this space. Hence one would like to 
have an analog of the Gaussian integral, i. e., Gaussian distribution 
functionals. 

A direct generalization of the Schwartz space to infinite dimensions 
meets difficulties. Indeed, if, for example, we define it as a space of 
weakly smooth functionals on an infinite dimensional (say, nuclear) 
space, with the norms of functionals analogous to the norms in the 
space S(M. n ), then a simplest functional 



(7 (it) is a linear functional, B(u, u) is a positive definite quadratic func- 
tional) is in general unbounded, and hence it does not belong to this 
space, which contradicts to requirement 5. 

Since Gaussian functions are important for us, it is natural to try 
to consider the finite dimensional case from the point of view of these 
functions. This leads to the Gaussian transform, which takes a function 
ip(xi, . . . , x n ) to the function 



The main purpose of the present paper is to describe the image of the 
Weil representation under the Gaussian transform. It turns out that 
it is rather easy to describe the image of the space S f (M n ) and of the 
space S"(lR n ) of tempered distributions, and it is less easy to describe 
the image of the space L2(M n ) and other spaces, cf. Remarks in §6. 
Thus, we obtain an explicit description of the Weil representation in 
the spaces S^R") and S"(IR ra ), in which the Gaussian vectors and the 
vector 1 play a distinguished role. This is the main result of the paper. 

The paper consists of 6 sections. §2 contains preliminaries on the 
Weil representation, §3 statements of main theorems, §§4,5 proofs, §6 
concluding remarks. 

A short exposition of the main theorems of the paper is contained in 



The author is grateful to V. V. Dolotin and Yu. A. Neretin for nu- 
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V(u) = l(u) exp(-B(u, u)) 
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2. The Weil representation 

2.1. The Weil representation arises when one solves the Schrodinger 
equation with a quadratic Hamiltonian: 

(3) 

^ = (L 2 a ^* + + ^"WjW* + T ^ 6 " J ^ 

Here ■?/>(£, x) is an unknown function of the variables t and x — (xi, . . . , x n ) 
djk and Cjk are real symmetric matrices; bjk is an arbitrary real matrix. 
The summand y J2j tyj is added to make the operator in the right 
hand side self-adjoint, and to simplify the formulas below. Put the 
initial condition 

(4) i;(0,x)=5(x-y). 

It turns out that exact solution of the problem (3,4) is given by the 
quasiclassical approximation. It reads [6] 

(5) ip(t, x) = a(t, h) exp(iS(t, x, y)/h), 

where S(t, x, y) is the action of the corresponding Hamiltonian system 
at the time t, a(t, h) is the amplitude, which are computed in the follow- 
ing way. The Hamiltonian flow at the time t gives a linear symplectic 

transformation of the phase space ( ^ ) , p — (pi, . . . ,p n ) T - We will 



x 

denote the points of the phase space by vector columns; the index T 
denotes transposing (in the present case, transposing of a row). Denote 

the matrix of this transform by ^ ^ ^ ^ e Sp(2n, R) (A,B,C,D are 

n x n- matrices), i. e., 

( q \ ( V \ P = Aq + By, 
\V J V x J ' x = Cq + Dy. 

Then 

(6) a(t,h) = l/^/(2irih) n detC, 

and the formula dS = pdx — qdy — H(x,p)dt (H(x,p) is the classical 
Hamiltonian) implies that 

(7) S(t, x, y) = X -x T AC- x x - y T C- l x + ^C^Dy. 



GAUSSIAN TRANSFORM OF THE WEIL REPRESENTATION 



5 



Below we put h — 1. Thus, the evolution operator of equation (3) at 
the time t has the form 

(8) 

y (27a) n det 6 J 

for det C ^ 0. The operator [/ is unitary and extends to a uni- 
tary operator on L 2 (M"). Indeed, this operator is the composition 
of four operators: 1) the operator of multiplication by the function 
exp ^y T C~ 1 Dy^, 2) the linear change of coordinates with the matrix 

(C T ) _1 , followed by multiplication by 1/Vdet iC, 3) the Fourier trans- 
form, 4) the operator of multiplication by the function exp (^x T AC~ 1 x) . 
All these operators are unitary. They also preserve the spaces S(M. n ) 
and 5"(R™). Besides that, they preserve the subspaces of even and odd 
functions, which we will denote by the indices respectively + and — , 
for example, S' + , (L 2 )-. 

It turns out and it is not difficult to check that the set of operators U 

given by formula (8) for all matrices ^ ^ ^ ^ with det C^O, can be 

extended to a two- valued representation of the group Sp(2n, R) in the 
spaces (L 2 )±, S±, S±, i. e., to a representation of a two-fold covering of 
the group Sp(2n, R). The two-valuedness is related to non-uniqueness 
of the square root from (2Tci) n det C. Let us call this covering by the 
metaplectic group, denoted Mp(2n, R). An explicit description of this 
group will be given in 2.2. 

This representation of the group Mp(2n, R) is usually called the Weil 
representation. 

The Lie algebra of the group Sp(2n, R) acts by the differential oper- 
ators 

(9) ^ \^a jkXj x k + b jkXj — + 2^—— j + - ^ b jr 

j,k J ' 3 

These operators are identified with the matrices from the symplectic 
Lie algebra by commuting with the operators 

(10) Yl y'r r J ■ "'''jjj-J ■ 

The operators (10) form a 2n-dimensional symplectic vector space V^; 
the symplectic form on Vi n is given by the commutator of operators. 
This implies (or one checks directly from (8)) that the operator U 
(8) conjugates operators (10) by the standard action of the matrix 
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A B 
C D 



on the space V 2n . In other words, if we write the coefficients 



v 



of operator (10) as a column vector y w Ji then conjugation by U 

takes operator (10) to the operator of the form (10) with the coefficients 
A B \ f v 
C D )\w 

This property determines the operator U uniquely up to a scalar 
factor. Indeed, if U' is another operator with the same property, then 
the operator U'U^ 1 commutes with the operators Xj and and 
hence it is multiplication by a constant, as it is not difficult to show. 
This implies once more that the operators U form a projective repre- 

(E B 
E 

acts by multiplication by the function exp (j 

)> the ma " 

trix ^ ^ ^ ^ ac ^ s ( U P ^° a constant factor) by Fourier transform, 

the matrix ^ ^ (A^y 1 ^ ac ^ s ^ com P os iti° n of a linear change of 

coordinates with the matrix A and multiplication by V det A. These 
matrices generate the group Sp(2n, R), which gives one more proof of 
existence of a projective action of Sp(2n, M) with the above described 
commutation relations with operators (10). 



2.2. Gaussian functions. These are functions 

(11) ipz{x) = exp I l - ^ ZjkXjXk I , 

V j,k J 

where Z is a symmetric complex matrix. We have ip z £ S' iff the 
imaginary part ImZ is nonnegative definite, ImZ > 0. Further, ip z € 
5 1 iff the matrix Im Z is positive definite, Im Z > 0. 

The action of the metaplectic group on the Gaussian functions is 
given by explicit formulas. This is seen from the fact that the function 
ipz satisfies the system of equations 

i-j^- + ^2 Z i kX ^j = 0, 1 < k < n. 
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A B 

The action of the matrix [ (j jj ) takes these equations to the equa- 



tions 



(12) ^ ( v jk%j + w i ki ^~) ^ = °> 1 - k - n ' 

3 

which, for det(CZ + D) ^ 0, are equivalent to the equations on the 
Gaussian function ^(az+b^cz+d)- 1 - 

The matrices Z with Im Z > form the so called Siegel upper half- 
plane ([4], §50), denoted SQ. This is a homogeneous space of the group 

Sp(2n,R); the matrix ^ ^ ^ ^ G Sp(2n, R) acts by a fractional lin- 
ear transform: 

Z ^ {AZ + B){C Z + D)- 1 . 

The stabilizer of the matrix Z = iE is the unitary group U(^)- 
An explicit calculation shows that 

(13) u ' h = vmmTW)' hAZ+B ' iCZ+D) -^ 

Define the metaplectic group Mp(2n, R) as the set of pairs 

(14) jQ,Vdet(CZ + £>)), 

where ^ ^ ^ ^ G Sp(2n,R), and ^det^Z + D) is one of the two 

continuous branches of the square root from det(CZ + D) ^ 0, Z G 
Then the multiplication law in Mp(2n, R) can be defined so that 
formula (13) will give a correctly defined single-valued action of this 
group on the functions ipz- 

Denote matrices Z with Im Z = by the letter a: 

(15) ip a (x) = exp ^ ^2 ajkXjX^j . 

The Mp(2n, R)-orbit of the function ip a consists, in general, of distri- 
butions. This orbit can be found as follows. The action of an element 
of Mp(2n, R) on the function ip a takes it to the function satisfying the 
system of equations (12) with real coefficients Vjk, Wjk- The operators 
in the left hand side of the system are linearly independent and pair- 
wise commute, i. e., form a basis of a Lagrangian subspace L in Vm- 
Conversely, for any real Lagrangian subspace L C Vi n there exists a 
unique, up to proportionality, distribution solution ^ of the system 
(12), in which the operators in the left hand side form a basis in L. It 
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is clear that the distribution ipL depends only on the subspace L but 
not on its basis. An example of ipi is the delta-function 8(x) satisfying 
the system of equations XkS(x) — 0, 1 < k < n. It is not difficult to 
see that in the general case ipi, is the product of a Gaussian function 
(15) on certain subspace in R n and the delta-function in the transversal 
direction. Let us emphasize once more that ipL is defined only up to a 
scalar factor. 

Thus, we have an embedding of the real Lagrangian Grassmannian 
A n into the projectivization of the space S', which takes L to CtpL- 
The image of this embedding is the Sp(2n, R)-orbit of the function 1. 
(Here one can see what is the form of the operator (8) for det C = 0: 
this is an integral operator with the Gaussian kernel supported on cer- 
tain subspace of the space (x, y) and corresponding to the Lagrangian 
subspace in R 2 ™ © R 2ra which is the graph of the symplectic trans- 



bundle ii on A n , whose fiber at the point L is the line CipL- This 
bundle is trivialized by the functions ip a (15) on the open dense chart 
of the Grassmannian parameterized by symmetric matrices a. Under 
change of chart and trivialization, given by the action of element (14) 
of Mp(2n, R), the corresponding transition functions can be found us- 
ing Maslov's method of canonical operator [6]. Indeed, the function 
ipL corresponds to the Lagrangian subspace L in the method of canon- 
ical operator. Under evolution of the Schrodinger equation (3), the 
function ip a goes to 



for some integer k (the Maslov index). The same formula is obtained 
in another way in the book [17], §21.6. For a third way of calculation 
of transition functions see below. 

The orbits of other functions ipz with Im Z > also, in general, 
consist of distributions. These orbits form the closure SQ of the Siegel 
upper half-plane in the complex Lagrangian Grassmannian. For L e 
SQ the function i/jl given by corresponding system (12) with complex 
coefficients, is in general the product of the Gaussian function ipz (11) 
on certain subspace in R ra , with Im Z > 0, and the delta-function in the 
transversal direction. Sp(2n, IR)-orbits are parameterized by the rank 
of the matrix Im Z on the subspace in IR n which is the support of the 
function ifj^. 

The line bundle /x extends to an Mp(2n, R)-equivariant line bundle 
Hi on the closure SQ, trivialized by the functions ipz over SQ. This 
and formula (13) imply a formula for the transition functions of the 




There arises an Mp(2n, R)-equivariant complex line 



^Aa + B)(Ca + D)-i • I det(Ca + D)\- 1 / 2 ■ e^ 2 
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bundle fi: the action of the element (14) corresponds to the transition 
function 



(16) lim l/y/det(CZ + D). 

Im Z^O 

(Cf. with the method of complex germ [7,8].) 

2.3. Gaussian transform. For a function ip(x) its Gaussian trans- 
form u(Z) = u^,{Z) is given by formula (2). Under the action of the 
group Mp(2n,R) on the function i]){x) its Gaussian transform u^(Z) 
transforms by an explicit formula, see Theorem 5 below. The main pur- 
pose of this paper is to describe the image of the Weil representation 
in the space S under the Gaussian transform. 

First let us find what obvious necessary conditions the function 
u^(Z) satisfies for ip e S. The function u^(Z) is well defined as a 
holomorphic function on the Siegel upper half-plane, which satisfies 
the following system of partial differential equations: 

(17) - d u - 9 — u 

dZji dZ km dZj m dZ k i 

where the operators for any j, k, are defined from the equalities 

(18) du = J^~ u dZ ik = Yl 2 lSr u dZ i k + 7^~ u dZ ir 

j,k jk j<k jk j " 



This system and the holomorphness condition are obtained by differen- 
tiation under the sign of the integral (2). For an odd function ip G S-, 
the function u^,{Z) vanishes. 

Further, the function u extends on the boundary of the Siegel upper 
half-plane as the section 

(19) u(L) = u4L) = (^,^ L ) 

of the line bundle //* dual to /ii. This section is continuous together 
with all its derivatives with respect to the action of the Lie algebra of 
Sp(2n, R). In particular, one defines an infinitely differentiable function 

(20) u(a) = u^(a) = J i^(x)e^ a ^ Xk dx, 
satisfying the system of equations 

(21) d d d d 

ddji ddkm ddjm ddkl 

It will be shown below that this system is invariant under the change 
of a chart and a trivialization by action of an element of Mp(2n, M). 
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Let us also define an analog of the Gaussian transform for odd func- 
tions ip(x) G This transform is given by the formula 



n. 



(22) ui{Z) = u l:i ,(Z) = J iP(x)xie^ z ^ Xk dx, 1<1< 

The holomorphic vector valued function (ui^(Z)) on the Siegel upper 
half-plane satisfies the system of equations 

< 23 > db (z) = 4 MZ) ' 

The vector valued function (ui^(Z)) extends to a section 

(2 J) llr.uAL) = //,,„•.,.(£) = ( )J ( + "V^J fa 



i v , w (L) = u v , w ^(L) = ^j, (vj x i + 



of the vector bundle W <g> on the closure of the Siegel upper half- 
plane. Here W is the tautological n-dimensional vector bundle over SQ, 
whose fiber over a point L e SQ is the space L itself or the isomorphic 
space (V2 n /L)*. The right hand side of (24) gives a linear functional 
on the space V 2n of vectors (v,w), vanishing on the subspace L, i. e., 
an element of (V 2 n/L)*. 

The section u VyW ^(L) is also continuous together with all derivatives 
with respect to the action of the Lie algebra of Sp(2n, R). In particular, 
one defines an infinitely differentiable vector valued function 

(25) m(a) = ui^(a) = J J{x)xie^ a ^ Xk dx, 1 < I < n, 

satisfying the system of equations 

/ \ ^ , N d . . 

(26) - — ui(a) = - — u k [a). 

da jk daji 

It will be shown below that this system is also invariant under change 
of a chart and a trivialization by action of an element of Mp(2n, M). 

3. Theorems 

The main theorem for even functions. 

Theorem 1. The transform (20) identifies the space S + with the 
space of smooth functions u(a) satisfying equations (21) and extending 
to sections of the bundle ji\ on the closure of the Siegel upper half- 
plane, which are holomorphic on the upper half-plane and continuous 
together with all derivatives with respect to the action of the Lie algebra 
of Sp(2n, M). The corresponding holomorphic function u(Z) on the 
Siegel upper half-plane automatically satisfies equations (17). 
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The analog for odd functions: 

Theorem 2. The transform (25) identifies the space S*_ with the 
space of smooth vector valued functions (ui(a)) satisfying equations 

(26) and extending to sections of the bundle W®n\ on the closure of the 
Siegel upper half-plane, which are holomorphic on the upper half-plane 
and continuous together with all derivatives with respect to the action 
of the Lie algebra of Sp(2n, R). The corresponding holomorphic vector 
valued function (ui(Z)) on the Siegel upper half-plane automatically 
satisfies equations (23). 

The following theorems are used in the proof of Theorems 1 and 2. 
They are more easy to prove, because they deal with distributions. 
The following theorem was stated as a conjecture by Yu. A. Neretin. 

Theorem 3. The transform (2) identifies the space S' + with the 
space of holomorphic functions u(Z) on the Siegel upper half-plane 
which satisfy equations (17) and have polynomial growth near the 
boundary of the half-plane, more precisely, which satisfy the estimate 

(27) \u(Z)\ < C(l + \Z\) M (1 + | detilmZ^'Y 

for some C, M, N. Here \Z\ is any norm on the space of matrices. 
The analog for odd distributions: 

Theorem 4. The transform (22) identifies the space S'_ with the 
space of holomorphic vector valued functions (ui(Z)) on SQ, whose 
components satisfy equations (23) and the estimate (27). 

Theorem 5. Under the identification from Theorem 3, the element 
inverse to the element (14) of Mp(2n, R) takes the function u(Z) to the 
function 

(28) v(Z) = 1 u((AZ + B)(CZ + D)' 1 ). 

A/det(GZ + u) 

Under the identification from Theorem 4 the same element takes the 

vector valued function (ui(Z)) to the function (vi(Z)), where 

(29) 

£((CZ + D) T U(Z) = 1 = u k ((AZ + B){CZ + Dy 1 ). 

j A/det(6z + D) 
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4. Proofs of Theorems 3,4,5 

4.1. Proof of Theorem 3. To make the Gaussian transform of a 
distribution ip(x), one should first construct its direct image under the 
map 

(30) x h-> b, b = (b jk ), b jk = xjx k , 

and then apply the Fourier-Laplace transform to the result. To prove 
Theorem 3, one should invert these transforms. For this it almost 
suffices to use more or less standard facts on distributions, exposed, for 
example, in [17], vol. 1. 

Let us first prove that for if) G S' + the function u^(Z) satisfies the 
estimate (27). For that let us represent the distribution if) in the form 
-^ipi(x), where if>\(x) is a continuous function of polynomial growth, 
a is a multiindex. It suffices to integrate (2) by parts, to eliminate J^, 
and estimate the obtained Gaussian integral, using the formula 

d f 

iXjX k ip z (x)dx = — — / if) Z {x)dx. 

O^jk J 

Let us now assume that the function u(Z) satisfies the conditions of 
the theorem, and let us find a function if> G S' + such that u = u$. 

Firstly, it is not difficult to show that the function u(Z) has a dis- 
tribution boundary value — the limit Uo(a) of functions u(a + iy) as 

n(n+l) 

functions of a as y — > 0, y > in the space S (R 2 ) of tempered 

n(ra+l) 

distributions on the space R 2 of real symmetric matrices a. 

Further, consider the inverse Fourier transform of the distribution 

uq. This is a distribution u e S"(R ( 2+ ) ) on the space R ( 2 " of real 
symmetric n x n-matrices b. The pairing between a and b is given by 
the formula (a, b) = | tr ab. It is not difficult to see that the function 
u(Z) is the Fourier-Laplace transform of the distribution u and that 
Uq is supported on the cone {b > 0} of nonnegative definite matrices, 
dual to the cone {y > 0}. By equations (17) the distribution u satisfies 
the equations 

(31) (bjibkm - b jm b k i)u = 0; 

in particular, it is supported on the cone of matrices of rank 1. 
Now Theorem 3 follows from the following Lemma. 

Lemma 1. The direct image under the map (30) identifies the space 

n(n + l) 

S^(R n ) with the space of distributions u G S"(R 1 ) supported on 
the cone {b > 0,rank& = 1} and satisfying equations (31). 
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Proof of Lemma 1 is based on the Borel theorem on construction 
of a smooth function with a given Taylor series at 0. This proof is 
elementary, and we omit it for shortness. 

Theorem 3 is proved. 

4.2. Proof of Theorem 4. It is not difficult to deduce Theorem 4 
from Theorem 3. Functions ui(Z) satisfying conditions of the theorem, 
also satisfy equations (17). By Theorem 3 one finds distributions ipi G 
S^(R n ) such that u\ = u^ r We deduce from equations (23) that x^ipi = 
xitpk for all k, I. This implies that ipi = xiip for some ip G S'_. This ip 
is the required one, q. e. d. 

4.3. Proof of Theorem 5. For u(Z) = u^(Z), ip G S' + (R n ), and for 
the element (14), denoted shortly by ^ ^ ^ ^ , we have 

< Z ) = I ( n n ) Mz^j = (j>>(c D^ z 

= ^ ^t(CZ + D)^ AZ+B)iCZ+Dr 

' -.u({AZ + B)(CZ + D)- 1 ) 



^/det(CZ + D) 

by formula (13). Similarly one deduces formula (29), but for com- 
putation of ^ ^ ^ ^ xiipz one should use the commutation relation 

between the element ^ ^ ^ ^ and the operator x\. Q. E. D. 

4.4. Remark. A direct computation shows that equations (17), (23) 
are invariant with respect to the actions (28), (29). Hence we obtain 
passing to the limit (16) that equations (21), (26) are invariant on the 
Grassmannian A n . 

5. Proofs of Theorems 1,2 

5.1. Proof of Theorem 1. The main idea of proof is the following. 
First, for a function ip G S + we express its L 2 -norm through the func- 
tion u^,(a), and then for arbitrary function u(a) satisfying conditions 
of the theorem, we prove that this expression is finite. This will imply 
that u = for some ip G L 2 . Similarly we prove that x a -^ip G L 2 for 
any multiindices a, f3. 
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Step 1. Let if) e S+. Let us express the L 2 -norm of ip through the 
function u(a) = u^(a). 

To this end, let us use the following argument related to the Fourier 
transform of a density on a submanifold in M N and taken from the proof 
of Theorem 7.1.26 in [17]. Denote bjk = XjXk for j < k, bjj = x 2 /2. 
Let us make the change of variables Xj — > b\j. We obtain 



/ 



(x)\ 2 dx = 2 J \i)(x)\ 2 dx 



(32) X1> ° 

1 } , f \Mbn,...Mn)\\ u 

= const • / — 2 dbu . . . db ln , 

J b n 



6ii>0 

where ipi(bn, ■ ■ ■ , b\ n ) = ip(x). Further, for bn > 0, k > j > 2 we have 
bjk = b lj b lk /2b n , bjj = b^/Abn, and 

i T, a jkbjk 

ip{x)e k - 3 dx 

f — T-, ; — r i ^ a]kh3k db n . . . db ln 

= const- / i/ji(bn,...,b ln )e k ^ — 2 , 

6i i 

bn>0 

whence 

(33) const .«(6y) = ( • • • > Me k ^- 2 ' ' IKi , hi > 0, 

( 0, fen < 0. 

Here u{b\j) is the inverse Fourier transform of the function u(a), con- 
sidered as a distribution in an, au, ■ ■ ■ , ai n with fixed for j, k > 2. 
Let us fix, once and for all, = — Sjk for j, k > 2, and denote an = s, 
a 12 = t 2 , a 13 = t 3 , . . ., a ln = t n , u(a) = u(s, t),t = (t 2 , . . . , t n ). Then 
(34) 

const((s+iO)-" /2 - 1 *M(s,t))~ = \ Mbn,...,b ln )e * 3 \ b u > 0, 

{ 0, On < 0. 

Here * denotes convolution with respect to the variable s; (s-MO) - ™/ 2-1 
is proportional to the Fourier transform of the function (bn) a J_ 2 . For- 
mulas (33) and (34) imply, by the Parcevale equality, that the L 2 -norm 
(32) of ip equals the integral 

(35) const • J u(s,t)((s + i0)" n/2_1 * u(s, t))dsdt. 

Step 2. Let us prove that for an arbitrary function u(a) satisfying 
conditions of the theorem, the integral (35) is absolutely convergent 
and nonnegative. 
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To this end, let us use the conditions at infinity following from the 
fact that u(a) extends to a smooth section of the bundle //* on the 
Grassmannian A n . Let us compute the change of trivialization of the 

section u by formula (16) for two matrices ^ ^ ^ ^: 1) the matrix 

corresponding to the Fourier transform in the variable x±, 2) the matrix 
corresponding to the Fourier transform in all variables xi, . . . , x n . The 
first one gives the formula 

(36) u(s, t) = s^^vis' 1 , tjs-^tjtks- 1 ), 
and the second one gives the formula 

(37) u(s, t) = (s + r 2 )- 1 /2 w (( s + r 2 )-\t 3 {s + r 2 )-\t 3 t k (s + r 2 )' 1 ), 

where r = \/t\ + t\ + . . . + t\\ v and w are smooth functions defined 
for all values of their arguments, including the zero values. 

Let us divide the integration domain of the integral (35) into three 
subdomains. 

1) \s\ < 2, r < 2. This domain is bounded, and the integral is finite 
in it. 

2) s < —2, r 2 < \s\ 1+e , where e > is sufficiently small (less than 
l/n). 

In this domain we have 

(38) \u(s,t)\ < const |s|~ 1/2 

by formula (36), since the arguments of the function v are bounded in 
this domain. Further, we have 

i 

\(s + i0)~ n/2 ~ l *u(s,t)\ < const J \s- s 1 \- n / 2 - 1 ds 1 

-l 

s-s 1 +20)- n/2 - 1 s^ 1/ %(sr 1 )^i 



+ const 

(39) 




< const Is]-^ 2 - 1 



< const Isl- 1 / 2 "™/ 2 . 



o 

n-l 

(s 2 -s- 1 +iO)- n / 2 ~ 1 s 2 2 v(s 2 )ds 2 



-1 



The latter inequality is obtained by integration by parts m times, where 
m = 7} for n even and m = '^-^ for n odd, and then, in the case of n 
even, by dividing the integral into the sum of three estimated integrals: 
from —1 to §s~\ from |s _1 to |s~\ and from |s _1 to 0. 
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Now the absolute value of the integral (35) over this domain is less 
than 



const • 



\s\- l - n/2 r n - 2 drds = const 



\s\ 2 2 as < oo. 



s<-2 
r 2 < | s |l+ e 



s<-2 



3) \s\ < 2, r > 2 or s > 2 or s < —2, r 2 > |s| 1+£ . In this domain, 
similarly to the estimates (38), (39), from formula (37) we deduce the 
estimates 

\u(s,t)\ < const(s + r 2 )~ 1/2 , 

|(S + tO)-^ 2 - 1 * U(s,t)\ < COnst (S + r 2 )-V2-n/2 ; 

and the absolute value of the integral over this domain is less than 
/ \ 



const 



| s |<2 s>2 s<-2 



\r>2 



< const r dr + 1 + / r dr 



oo 

+ J (^r 2 -2)- n ' 2 - (r'-r^y^ 

l+e 



r n ~ 2 dr 



< oo. 



J 



Nonnegativity of the integral (35) follows from the fact that it can 
be presented in the form 



const 



j \(s + iO) 



-n/4-l 



* u(s, t)\ 2 dsdt, 



at least for a function u(a) with compact support. The general case is 
reduced to this one by multiplication of the function u(a) by a smooth 
function < p(a) < 1 with compact support, equal to 1 in a large 
neighborhood of the origin. 

Step 3. Let u(a) be a function satisfying the conditions of the 
theorem. Let us prove that u = for some ip G S' + . 

Due to formula (16) for the transition function of the bundle /i, the 
function u(a) is bounded and, moreover, it tends to zero as 0( \ det(Ca+ 
D)\~ l l 2 ) as a — > oo in any direction, for appropriate matrices C and D. 
Therefore, the function u(Z) is also bounded, because the closure of 
each Sp(2n, M)-orbit is compact, and one can use induction with respect 
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to the dimension of an orbit. Hence u(Z) coincides with the Fourier- 
Laplace transform of u (see proof of Theorem 3). Hence the extension 
u(Z) is unique. Besides that, u(Z) satisfies equations (17). Indeed, the 
difference of the left and right hand sides of these equations is defined 
as a section of the bundle //* on SQ (because the operator belongs 

to the Lie algebra of Sp(2n, M.)) and vanishes on the Grassmannian A n . 
Hence by Theorem 3 there exists a unique distribution ip e S' + such 
that u = u$. It remains to prove that ip & S + . 
Step 4. Let us prove that ip G L 2 . 

Denote the Hermitian inner product on the space of functions u 
satisfying the conditions of the theorem, given by the integral (35), by 
( , ) . Consider a linear functional on the space of functions ip & S + 
given by the formula 

(40) y? h-> (uf,^). 

This functional can be expressed through ip and ip in the following way. 
Assume that the function tp(x) has compact support and vanishes in a 
neighborhood |xi| < e of the hyperplane x\ = 0. Then 



Indeed, in this case the function <p can be expressed as a smooth func- 
tion ipi(bij) of &n, . . . , bi n , vanishing for bu < e 2 /2. The number (if), tp) 
equals the pairing (^i, ipi) with the direct image ipiib) of the distribu- 
tion ip to the space of variables bj k . This pairing is well defined since 
the intersection of supports of the functions ipiib) and (pi(b) is compact. 
It also equals to the pairing (ip 2 , ^2) , where 



and a\ k = a° kl = 0, a° fe = —Sjk for j, k > 2, as fixed above; = 
bijb lk /2b n for k > j, = bfj/Abn, as above. 

This pairing is equal to the pairing (ip 3 , ip 3 ), where 1^3(0) (respectively 
<f3{a)) is the Fourier transform of the distribution ip2(b) (respectively 
<P2(b)). We have 



(41) 




tp 3 (a) = u^(a + a ), 

y? 3 (a) = const((s + i0)"" /2 " 1 



* u. 



(s,t))- H 5{a ]k ) 



k>j>2 



18 



A. V. STOYANOVSKY 



(see (34)). This implies (41). 

Furthermore, by the Cauchy inequality for the scalar product ( , ) , 
the functional (40) is continuous with respect to the L 2 -norm of the 
function tp. This implies that ip equals to the sum of an L 2 -function 
and a distribution supported at the hyperplane x\ — 0. 

Applying the same argument to other variables Xj instead of x\ and 
to other columns of the matrix bjk, we obtain that ip equals to the sum 
of an L 2 -function and a distribution supported at the point x — 0. 

Now consider the Fourier transform ip of the function if). It equals 
to the sum of an L 2 -function and a polynomial. On the other hand, 
the Fourier transform (up to a constant factor) belongs to the action 
of the metaplectic group, and this action preserves the space of func- 
tions u satisfying the conditions of the theorem. Applying the previous 
argument to the function instead of u, we obtain that ip is also the 
sum of an L 2 -function and a distribution supported at the point x — 0. 
Therefore, ip G L 2 , as required. 

Step 5. Now let us apply the previous argument to the distributions 
x a -^ip. Since the operators tXjXk, x j~^ + \^jki an d ^af"gf^ belong 
to the Lie algebra of Sp(2n, M.) (see (9)), and this action preserves the 
space of functions u satisfying conditions of the theorem, we obtain 
that the distribution x a -^ip belongs to L 2 for any a, (3 with even 
|a| + \P\. From the inequality \xj\ < (1 + x?)/2 we conclude that the 
same is true for odd |a| + \/3\. 

Therefore, ip G S + . 

Theorem 1 is proved. 

5.2. Proof of Theorem 2. Theorem 2 is deduced from Theorem 1 in 
the same way as Theorem 4 was deduced from Theorem 3. Q. E. D. 

6. Remarks 

1) Similar formulas, equations, and constructions appear in the the- 
ory of the Radon transform due to I. M. Gelfand and others, cf. [1], §7 
and references there. 

2) A close description of the spinor representation of the orthogonal 
group is contained in the book [12], Ch. 12. 

3) The condition of being extended to the Siegel upper half-plane 
(and not only to the real Lagrangian Grassmannian) in Theorem 1 is 
essential, as one can see already in the case n — 1. Indeed, consider a 
function u{a) of one variable with compact support. If the function u(a) 
cannot be extended to the upper half-plane, then its inverse Fourier 
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transform is not supported on the positive real axis, and it is easy to 
see that the corresponding function ip(x) does not exist. 

4) The question how to describe the Gaussian transform of the space 
L 2 seems not easy. The construction of reproducing kernel (cf. [9], §8) 
gives only a non-explicit description of the Gaussian transform of the 
space L 2 as the image of an invariant integral operator on the Siegel 
upper half-plane. This question is related to the following problem: 
what is the invariant formula for the pairing between the spaces S' 
and S under the identifications from Theorems 1-4? In particular, this 
would give an invariant formula for the L 2 -norm, which could simplify 
the proof of Theorem 1. Formally it is not hard to write such a formula: 



(42) const • J u{Z)u{Z) 

sg 



dZdZ 



(det ImZ) 1 



It is more difficult to give a sense to it. Seemingly, it should be done by 
analytical continuation with respect to the exponent n + \. Cf. [11]. 

5) Under the identifications from Theorems 1-2, the topology of the 
space S goes to the C°°-topology on the space of sections of a bundle 
on the Grassmannian A n . This can be easily deduced from the proof 
of Theorems 1-2 (§5). Convergence in this topology implies uniform 
convergence of holomorphic functions on the Siegel upper half-plane 
and C°°-convergence on any Sp(2n, M)-orbit in the closure of the upper 
half-plane. 

6) The group Mp(2n, M) acts also in the Gelfand-Shilov space Syl(E, n ) 

1 /2 

[2] and in the dual space (S^)'(R n ). One can ask whether the Gauss- 
ian transform of the space sH% (respectively (Sy^)') coincides with 
the space of analytical sections of bundles [/,*, W <S> H* on the real 
Lagrangian Grassmannian, which satisfy equations (21), (26) and can 
be analytically continued to the closure of the Siegel upper half-plane 
(respectively, with the space of all holomorphic solutions of equations 
(17), (23) on the Siegel upper half-plane). It is interesting whether 
there exist other functional spaces defined simply enough with an ac- 
tion of the metaplectic group. It is known that the space S coincides 
with the space of smooth vectors in L 2 in the sense of representation 
theory of the group Mp(2n, M.) [9]. Does the space sH% coincide with 
the (appropriately defined) space of analytical vectors in L-p. 

7) On infinite dimensional generalizations. The expected infinite 
dimensional generalization of the construction of the space S+ from 
Theorem 1 can be performed in the following setup. Instead of the 
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space R n , one considers a real nuclear space V, say, the Schwartz space. 
Instead of the symplectic space R 2n , one considers the space V © V* 
with its natural symplectic form. Instead of the metaplectic group, 
one considers a central extension of the group of continuous symplectic 
automorphisms of the space V © V*. The Lie algebra cocycle of this 
central extension is given by 



Instead of determinant, one considers the Fredholm determinant (cf. 
[20]). Instead of symmetric (positive definite) matrices, one considers 
symmetric continuous (positive definite) operators V — > V* or V* — > V. 

This way one can seemingly obtain a space analogous to S + and 
suitable for solution of equations of the form 



where b(x, y) is a distribution, c(x, y) is a smooth function, and a(x, y) 
is the kernel of a linear operator V — > V. But this equation does not 
look like equation (1), in which both b and c are singular distributions. 
In addition, the evolution operators of the Klein-Gordon equation do 
not belong to this infinite dimensional symplectic group. They are 
rather continuous operators V © V — > V © V, where V is the Schwartz 
space. 

Hence we come to a conclusion that this approach to equation (1) is 
not fruitful. For a more fruitful approach, see [18]. 
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